Join Cost for Unit Selection Speech Synthesis

In unit-selection speech synthesis systems, synthetic speech is produced by concatenating speech units selected from a large database, or inventory, which contains many instances of each speech unit with varied prosodic and spectral characteristics. Hence, by selecting an appropriate sequence of units, it is possible to synthesize highly natural-sounding speech. The selection of the best unit sequence from the database is typically treated as a search problem in which the best sequence of candidates from the inventory is the one that has the lowest overall cost [1]. This cost is often decomposed into two costs: a target cost (how closely candidate units in the inventory match the specification of the target phone sequence) and join cost (how well neighboring units can be joined) [1]. If, as is usually the case, the cost functions used to compute these costs take into account only properties of the fixed target sequence and local properties of the candidates, the optimal unit sequence can be found efficiently by a Viterbi search for the lowest cost path through the lattice of the target and join costs. In this chapter we focus on the calculation of the join cost (also known as concatenation cost). The ideal join cost is one that, although based solely on measurable properties of the candidate units—such as spectral parameters, amplitude, and F0—correlates highly with human listeners’ perceptions of discontinuity at concatenation points. In other words, the join cost should predict the degree of perceived discontinuity. We use this terminology: a join cost is computed using a join cost function, which generally uses a distance measure on some parameterization of the speech signal.